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Abstract 

We revisit Levin’s theory about the 
correspondence of verb meaning and 
syntax and infer semantic classes from 
a large syntactic classification of more 
than 600 German verbs taking clausal 
and non-finite arguments. Grasping the 
meaning components of Levin-classes is 
known to be hard. We address this chal¬ 
lenge by setting up a multi-perspective 
semantic characterization of the inferred 
classes. To this end, we link the inferred 
classes and their English translation 
to independently constructed semantic 
classes in three different lexicons - the 
German wordnet GermaNet, VerbNet 
and FrameNet - and perform a detailed 
analysis and evaluation of the resulting 
German-English classification (avail¬ 
able at WWW. ukp . tu-darmstadt. 
de/modality-verbal asses/). 

1 Introduction 

Verbs taking clausal and non-finite arguments add 
a further meaning component to their embedded 
argument. For example, the embedded argument 
is realized as f/iaf-clause in (1) and (2), but un¬ 
derstand in (1) marks it as factual and hope in (2) 
as uncertain. The verb pretend in (3) realizes its 
embedded argument as non-finite construction and 
marks it as non-factual. 

(1) He understands that his computer has a 
hardware problem. 

(2) She hopes that her experience will help 
others. 

(3) He pretends to take notes on his laptop, but 
really is updating his Facebook profile. 


The entities expressed by embedded clausal and 
non-finite arguments are also called “abstract ob¬ 
ject” (AO) in the rest of this paper (follow¬ 
ing Asher (1993)); we will use the linguistic 
term “modality” (Hacquard, 2011) to subsume the 
meanings (such as factuality, non-factuality and 
uncertainty) denoted by AO-selecting verbs. 

As AO-selecting verbs can change the mean¬ 
ing of a text in important ways, text understand¬ 
ing systems should be sensitive to them. In par¬ 
ticular, classifications of AO-selecting verbs ac¬ 
cording to semantic criteria are important knowl¬ 
edge sources for a wide range of NEP applications, 
such as event tagging (Saurt et al., 2005), com- 
mited belief tagging (Prabhakaran et al., 2010), 
reported speech tagging (Krestel et al., 2008), 
the detection of uncertainty (Szarvas et al., 2012) 
and future-oriented content (Eckle-Kohler et al., 
2008), textual entailment (Saurt and Pustejovsky, 
2007; Eotan et al., 2013), or determining the 
degree of factuality of a given text (Saurt and 
Pustejovsky, 2012; de Mameffe et al., 2012). 
Accordingly, various semantic classifications of 
AO-selecting verbs have been developed, e.g., 
(Kiparsky and Kiparsky, 1970; Karttunen, 1971; 
Karttunen, 2012), some of them explicitly in the 
context of NEP (Naim et al., 2006; Saurt, 2008). 

However, these classifications are constructed 
manually and often quite limited in coverage. 
Consequently, extending or adapting them to spe¬ 
cific domains or other languages is a major issue. 

We propose to address this issue by exploiting 
the relationship between the syntactic behavior of 
verbs and their meaning following Eevin’s theory 
(Eevin, 1993). This has not been done yet for 
verbs signaling modality, as far as we are aware. 
For the particular category of AO-selecting verbs, 
Eevin’s theory allows constructing verb classifi¬ 
cations in a purely syntax-driven way, i.e. in¬ 
ducing semantic classes from syntactically defined 





classes, and thus possibly also extending given 
classes using large corpora. ^ 

While the appeal of Levin’s hypotheses is clear, 
we are aware of a major difficulty, making our 
approach a challenging research problem: it is 
very hard to grasp the precise meaning compo¬ 
nents which are to be associated with a syntactic 
“Levin” class. At the same time, it is vital to have 
a good semantic characterization of the meaning 
components in order to apply such classes to NLP 
tasks in an informed way. 

We address these issues and make the follow¬ 
ing contributions: (i) We consider a purely syn¬ 
tactic classification of more than 600 German AO- 
selecting verbs and induce semantic classes based 
on findings from formal semantics abouf corre¬ 
spondences befween verb synfax and meaning. 
This yields an inifial description of fhe mean¬ 
ing componenfs associafed wifh fhe classes, along 
wifh a fenfalive class name, (ii) In a second sfep, 
we refine and exfend fhe semanfic characferiza- 
fion of fhe verb classes by franslafing if fo English 
and linking if fo exisfing semanfic classes in lexi¬ 
cal resources af fhe word sense level: we consider 
fhe coarse semanfic fields in fhe German wordnef 
GermaNef (Kunze and Lemnifzer, 2002), fhe verb 
classes in fhe English lexicon VerbNef (Kipper el 
ah, 2008), and fhe semanfic frames in fhe English 
lexicon ErameNef (Baker el ah, 1998). As a re- 
sull, we oblain a delailed semanfic characleriza- 
lion of fhe verb classes, as well as insighls info 
fhe validity of Eevin’s Iheory across fhe relaled 
languages German and English, (iii) We also per¬ 
form a lask-orienfed evalualion of fhe verb classes 
in fexlual enfailmenf recognilion, making use of 
insighls from fhe previous Iwo steps. The resulls 
suggesl lhal fhe verb classes mighl be a promising 
resource for Ibis lask, for German and for English. 

2 Related Work 

This seclion summarizes related work abouf fhe 
correspondence befween verb meaning and synfax 
and discusses related work on modality in NEP. 

Syntactic Reflections of Verb Meaning Se¬ 
mantic verb classifications lhaf are grounded in 
lexical-synlaclic properlies of verbs are parlicu- 
larly appealing, because Ihey can aulomalically 
be recovered in corpora based on synlaclic fea- 
lures. The mosl well known verb classificalion 

'Abstract objects already characterize the possible seman¬ 
tic roles to a certain extent. 


based on correspondences befween verb synfax 
and verb meaning is Eevin’s classification (Eevin, 
1993). According fo Eevin (2015a), verbs lhal 
share common synlaclic argumenl allernalion paf- 
lerns also have parlicular meaning componenfs in 
common, Ihus Ihey can be grouped info a seman¬ 
tic verb class. Eor example, verbs participating in 
fhe dative alternation^ can be grouped info a se¬ 
mantic class of verbs sharing fhe particular mean¬ 
ing componenl “change of possession”, Ihus Ibis 
shared meaning componenl characlerizes fhe se¬ 
mantic class. Recenf work on verb semanlics pro¬ 
vides additional evidence for Ibis correspondence 
of verb synfax and meaning: Harlshorne el al. 
(2014) reporl lhal fhe synlaclic behavior of some 
verbs can be predicled based on Iheir meaning. 

VerbNef is a broad-coverage verb lexicon orga¬ 
nized in verb classes based on Eevin-slyle syn¬ 
laclic allernalions: verbs wifh common subcal- 
egorizalion frames and synlaclic allernalion be¬ 
havior lhal also share common semanfic roles are 
grouped info VerbNef classes. VerbNef no! only 
includes fhe verbs from fhe original verb classifi¬ 
cation by Eevin, bul also more lhan 50 addilional 
verb classes (Kipper el al., 2006) aulomalically 
acquired from corpora (Korhonen and Briscoe, 
2004). These classes confain many AO-selecling 
verbs lhal were nol covered by Eevin’s classifica¬ 
tion. However, VerbNef does nol provide infor- 
malion abouf fhe modal meaning of AO-selecling 
verbs and does nol reflecl fine-grained dislincfions 
befween various kinds of modality. 

There is also some criticism in previous work 
regarding fhe validily of Eevin’s approach. Baker 
and Ruppenhofer (2002) and Schnorbusch (2004) 
bolh discuss various issues wifh Eevin’s original 
classificalion, in particular fhe difficully fo grasp 
fhe meaning componenfs, which are fo be associ¬ 
ated wifh a Eevin class. 

While approaches fo exploif fhe synlaclic be¬ 
havior of verbs for fhe aulomafic acquisilion of se¬ 
mantic verb classes from corpora have been de¬ 
veloped in fhe pasl, Ihey were used fo recover 
only small verb classifications: Schulle im Walde 
(2006)’s work considered a semanlically balanced 
sel of 168 German verbs, Merlo and Stevenson 
(2001) used 60 English verbs from fhree parlicular 
semanfic classes. 

In conlrasl fo previous work, we consider a large 

^These verbs can realize an argument syntactically either 
as noun phrase or as prepositional phrase with to. 



set of more than 600 German AO-selecting verbs 
and focus on their modal meaning (i.e., expressing 
factuality or uncertainty). 

Related Work on Modality in NLP Previous 
work in NLP on the automatic (and manual) anno¬ 
tation of modality has often tailored the concept of 
modality to particular applications. Szarvas et al. 
(2012) introduce a taxonomy of different kinds of 
modality expressing uncertainty, such as deontic, 
bouletic, abilitative modality, and use it for detect¬ 
ing uncertainty in an Information Extraction set¬ 
ting. Their uncertainty cues also include verbs. 

Sauri and Pustejovsky (2012) use discrete val¬ 
ues in a modality continuum ranging from uncer¬ 
tain to absolutely certain in order to automatically 
determine the factuality of events mentioned in 
text. Their automatic approach is based on the 
FactBank corpus (Sauri and Pustejovsky, 2009), a 
corpus of newswire data with manually annotated 
event mentions. For the factuality annotation of 
the event mentions, the human annotators were in¬ 
structed to primarily base their decision on lexi¬ 
cal cues. For example, they used verbs of belief 
and opinion, perception verbs, or verbs expressing 
proof. 

Nissim et al. (2013) introduce an annota¬ 
tion scheme for the cross-linguistic annotation of 
modality in corpora. Their annotation scheme de¬ 
fines two dimensions which are to be annotated 
(called layers): factuality (characterizing the em¬ 
bedded proposition or concept) and speaker’s at¬ 
titude (characterizing the embedding predicate). 
Their annotation scheme starts from a fixed sef of 
modal meanings and aims al finding previously 
unknown Iriggers of modalily. However, some 
modal meanings are nol distinguished, in parlic- 
ular fhose involving fulure-orienlalion. A classi¬ 
fication approach grounded in synlax - as in our 
work - can be considered as complemenlary: if 
slarls from Ihe synlaclic analysis of a large sef 
of frigger words, and induces a broad range of 
modal meanings based on correspondences be- 
fween verbs synfax and meaning. 

Our semanfic classificalion for AO-selecfing 
verbs covers a wide range of differenf kinds of 
modalily in lexl, Ihus considerably exlending pre¬ 
vious work. 

3 Inferring Semantic Verb Classes 

In Ibis seclion, we infer semanfic verb classes from 
Ihe synlaclic allernalion behavior of a large dalasel 


of German AO-selecling verbs. The research hy- 
polheses underlying our melhod can be summa¬ 
rized as follows: There are correspondences be- 
Iween verb synlax and meaning: cerlain synlac- 
lic alternations correspond lo particular meaning 
componenls (Fevin, 2015a). 

3.1 German Subcategorization Lexicon 

We consider a sel of 637 AO-selecting verbs given 
in (Eckle-Kohler, 1999). These verbs are a subset 
of a subcategorization lexicon (i.e., pairs of lemma 
and subcategorization frame) that has automati¬ 
cally been extracted from large newspaper cor¬ 
pora using a shallow regular expression grammar 
covering more than 240 subcategorization frames 
(short: subcat frames). All the subcat frames ex¬ 
tracted for a given verb were manually checked 
and only the correct ones were included in the fi¬ 
nal lexicon, because high quality lexical informa¬ 
tion was crucial in the target application Fexical 
Functional Grammar parsing.^ 

Eckle-Kohler (1999) specified the alternation 
behavior of each AO-selecting verb regarding dif¬ 
ferent types of clausal and non-finite arguments, 
yielding a syntactic signature for each verb (e.g., 
111101 for the verb einsehen (realize) using the 
encoding in Table 1, top to bottom correspond¬ 
ing to left to right).^ For this, each verb was in¬ 
spected regarding its ability to take any of the con¬ 
sidered clausal and non-finite constructions as ar¬ 
gument - either on the basis of the automatically 
acquired subcat frames or by making use of lin¬ 
guistic introspection. Finguistic introspection is 
necessary to reliably identify non-possible argu¬ 
ment types, since missing subcat frames that were 
not extracted automatically are not sufficient as ev¬ 
idence. 

Although there are 64 possible syntactic signa¬ 
tures according to basic combinatorics, in the data 
only 46 signatures were found, which group the 
verbs into 46 classes. While Eckle-Kohler (1999) 
points out a few semantic characteristics of these 
classes, most of them lack a semantic characteri¬ 
zation. Our goal is to address this gap and to in¬ 
fer shared meaning components for all the classes. 

^Today, this lexicon is part of the larger resource “IMSLex 
German Lexicon” (Fitschen, 2004). 

"^The automatically extracted subcategorization lexicon 
also contains adjectives and nouns taking clausal or infinitival 
arguments. However, many of the 1191 nouns and 666 adjec¬ 
tives are derived from verbs, which makes them the central 
word class. 



Argument Type 

Y/N 

Example 

dafit Aafi-clause 

1/0 

sehen (see) 

zt<(fo)-infinitive, present 

1/0 

versuchen (try) 

ZM(to)-infinitive, past 

1/0 

bereuen (regret) 

vrh-clause 

1/0 

einsehen (realize) 

ob( whether/if)-c\sMse 

1/0 

fragen (ask) 

declarative clause 

1/0 

schreien (shout) 


Table 1: Clausal and infinitival arguments distin¬ 
guished in the syntaetie elassifieation; possibility 
of eaeh type is eneoded as 1 (possible) or 0 (not 
possible). 

For this, we use linguistie researeh findings as de- 
seribed in the next seetion. 

3.2 Findings from Formal Semantics 

We employ the following findings on eorrespon- 
denees between verb meaning and syntax in order 
to infer semantie elasses from the syntaetie signa¬ 
tures. This gives also rise to tentative names (la¬ 
bels) for the eorresponding meaning eomponents. 

Factuals: the that-wh and the that-wh/if al¬ 
ternation. Verbs that are able to alternatively take 
that and w/i-elauses eoeree the embedded interrog¬ 
ative and deelarative elauses into faetual AOs, eor¬ 
responding to a partieular faet (Ginzburg, 1996). 
Among the verbs showing the that-wh alternation 
are the well-known faetive verbs (Kiparsky and 
Kiparsky, 1970) (e.g.. She proves that she exists. 
vs. She proves who she is. vs. He proves whether 
he can mine gold.). 

There is a further distinetion among these 
verbs regarding the ability to take an embedded 
///w/ief/ier-question: Sehwabe and Fittler (2009) 
show that the that-wh/if alternation is eonneeted 
to objeetive verbs entailing the existenee of an in¬ 
dependent witness, whereas the that-wh alterna¬ 
tion (i.e., an ///w/iet/ier-question is not possible) 
oeeurs with non-objeetive verbs (e.g.. He regrets 
whom he ended up with. vs. ^He regrets whether 
he ended up playing this game.). 

“Aspectuals”: the inability to take that- 
clauses and to-infinitives in the past tense. 
Recently, linguistie researeh has increasingly 
addressed particular semantic aspects of to- 
infinitives. Kush (2011) has investigated AOs that 
can neither be realized as toaf-clause nor as to- 
infinitive in the past tense (e.g.. She hesitates to 
answer, vs. *She hesitates to have answered.^ vs. 

^This is the literal translation of the German equivalent to 
English. In English, the ing-form in the past would be more 


★S'/ie hesitates that...) These AOs are selected by 
control verbs^ and can be characterized as men¬ 
tal actions. Kush (2011) points out that the verbs 
selecting those AOs have an aspectual meaning in 
common. 

Future orientation: to-infinitives in the 
present tense and the inability to take to- 
infinitives in the past tense. Laca (2013) has in¬ 
vestigated verbs across English and Spanish that 
embed future-oriented AOs. Only future-oriented 
AOs can be used with future-oriented adverbials, 
such as tomorrow, and these AOs are often real¬ 
ized as non-finite constructions, e.g., to-infinitives. 
She points out that not only control verbs take 
future-oriented AOs, but also verbs expressing at¬ 
titudes of preference. This finding implies that 
such future-oriented AOs are typically incompati¬ 
ble with past-oriented adverbials (e.g., yesterday) 
and verb forms in the past tense (e.g., i^She plans 
having finished the assignment yesterday.). 

3.3 Mapping to Meaning Components 

We automatically infer semantic classes based on 
a manually constructed mapping between the syn¬ 
tactic signatures from Eckle-Kohler (1999) and the 
meaning components grounded in syntax summa¬ 
rized in Section 3.2.^ 

We constructed this mapping in two steps: In a 
first step, the signatures are aligned to the meaning 
components from Section 3.2 based on substrings 
of the signatures: future-orientation matches the 
110 prefix, aspectual the 010 prefix, and factual- 
ity matches I’s in fourth or fifth position. It is 
important to point out that future-orientation can 
be combined with factuality: this corresponds to 
an independent matching of the 110 prefix and the 
factuality substring. While this combination may 
seem contradictory, it reflects the lexical data and 
shows that also weak forms of factuality (“it will 
most likely be factual at some point in the future”) 
are expressed in language. 

In a second step, the pre-aligned signatures are 
merged, if the remaining slots of the signature are 
either 1 or 0 (i.e. the respective argument types 
can or can not occur); in the resulting merged sig- 

typical instead of a fo-infinitive in the past tense. 

^“Control” refers to the co-reference between the implicit 
subject of the infinitival argument and syntactic arguments in 
the main clause, either the subject (subject control) or direct 
object (object control). 

*We did not consider verbs can be used with all kinds 
of clausal and infinitival arguments, such as the majority of 
communication verbs (e.g., comment, whisper). 



signature #verbs - examples 

1 meaning components 

1 semantic characterization (#llnked verbs) 

010 — 

36 (6%) - wagen (dare), 
zogern (hesitate), weigern 
(refuse) 

aspectual: verbs expressing 
the ability of doing an action 

VN (2): consider-29.9, wish-62; FN (2): pur¬ 
pose, cogitation 

110 0- 

195 (31%) - anbieten 

(offer), empfehlen (recom¬ 
mend), fordern (demand) 

future-oriented: verbs 

marking AOs as anticipated, 
planned 

VN (89): force-59, forbid-67, wish-62, promote- 
102, urge-58.1, order-60, admire-31.2, order-60, 
promise-37.13 ; FN (43): request, preventing 

000 11- 

15 (2%) - nachfragen (in¬ 
quire), anfragen (ask) 

interrogative: verbs mark¬ 
ing AOs as under investiga¬ 
tion 

VN (3): estimate-34.2, inquire-37.1.2, order-60; 
FN (1): questioning, request 

111 1- 

122 (19%) - bedauern (re¬ 
gret), uberwinden (over¬ 
come), danken (thank) 

wh-factual: opinion verbs 
marking AOs as factual 

VN (45): transfer-mesg-37.1.1, wish-62, 

admire-31.2, complain-37.8, conjecture-29.5, 
say-37.7; FN (18): statement, reveal-secret 

110 10- 

30 (5%) - befurworten 
(approve), verteidigen (de¬ 
fend), loben (praise) 

future-oriented wh-factual: 

opinion verbs marking AOs 
as future-oriented and factual 

VN (15): admire-31.2, allow-64, transfer-mesg- 
37.1.1, suspect-81, characterize-29.2 , neglect- 
75, want-32.1, defend-85, comprehend-87.2; 
FN (10): judgment, grant-permission, defend, 
experiencer-focus, judgment-communication, 
justifying, hit-or-miss, statement, reasoning, 
tolerating, grasp 

1- 11- 

120 (19%) - beschreiben 
(describe), horen (hear), 
erinnern (remember) 

wh/if -factual: objective 
verbs marking AOs as factual 

VN (55): discover-84, say-37.7, see-30.1, 
comprehend-87.2, rely-70, seem-109, consider- 
29.9, transfer-mesg-37.1.1, estimate-34.2, 
inquire-37.1.2; FN (23): perception-experience, 
statement, cogitation, grasp 

110 11- 

48 (8%) - festlegen (deter¬ 
mine), abschdtzen (assess), 
lehren (teach) 

future-oriented wh/lf- 

factual: objective verbs 

marking AOs as future- 
oriented and factual 

VN (28): estimate-34.2, rely-70, indicate- 
78, transfer-mesg-37.1.1, correspond-36.1, 
conjecture-29.5, discover-84, say-37.7; FN 
(16): predicting, education-teaching, assessing, 
reliance, reasoning 

111 0- 

66 (10%) - vorwerfen 

(accuse), bestreiten (deny), 
fiirchten (fear) 

non-factnal: verbs marking 
AOs as not resolvable re. 
their factuality 

VN (28): conjecture-29.5, wish-62, complain- 
37.8, admire-31.2; FN (13): statement, reveal- 
secret, experiencer-focus, certainty 


Table 2: The 632 verbs in 8 semantie elasses (5 verbs show idiosyneratie behavior). Signature sub¬ 
strings in bold eorrespond to meaning eomponents, whieh (along with tentative elass names) are based 
on See. 3.2. The eross-lingual semantie eharaeterization shows aligned VerbNet (VN) elasses eovering 
265 (42%) verbs and aligned FrameNet (FN) frames eovering 126 (20%) verbs, see See. 4.1.® 


nature, these slots are left underspeeified. Merging 
the signatures in this way yields 8 partially under¬ 
speeified signatures whieh eorrespond to the final 
semanfie elasses. This proeedure eovers more fhan 
99% of fhe 637 verbs under invesfigafion: only 
5 verbs showed idiosyneratie synfaelie behavior, 
4 of fhose are verbs fhaf ean lake an AO as sub- 
jeel (e.g., bedeuten (mean)). As a eonsequenee of 
fhe aulomalie pari of Ihis proeedure, every verb 
is assigned lo exaefly one elass - a simplifiealion 
whieh we plan lo resolve as pari of fulure work. 

Table 2 provides an overview and a eharaeleri- 
zalion of Ihese elasses, also showing Ihe final sig- 
nalures and Iheir subslrings whieh eorrespond lo 
Ihe meaning eomponenls. The non-faelual elass is 
derived from Ihe wh-faelual elass: Ihe only differ- 
enee is Ihe inabilily lo lake a w/i-elause (e.g. * He 
hopes, when he will succeed.). 


While Ihe deseriplions of Ihe meaning eompo¬ 
nenls and Ihe elass names are inspired from re- 
seareh in linguislies (lypieally a very deep anal¬ 
ysis of only few verbs), Iransferring Ihem lo our 
verb resouree - whieh is of mueh larger seale - in- 
evilably leads lo oullier verbs in Ihe elasses, e.g., 
verbs lhal do nol slrielly maleh Ihe elass label. 
Examples inelude verbs sueh as Uberlegen (con¬ 
sider) in Ihe wh/if-factual elass (nol eovering Ihe 
fulure-orienled meaning eomponenl) or schaden 
(harm) as non-factual rather lhan as wh-factual. 
For Ihis reason, and also beeause of Ihe assignmenl 
of highly polysemous verbs lo only one elass, Ihe 
definitions of meaning eomponenls and Ihe elass 
names should ralher be eonsidered as loose, pro¬ 
viding a lirsl lenlalive semantie eharaeterization of 
Ihe modalily elasses. 

In sum, Ihis seelion presented an inventory 
























of modal meaning components that we primar¬ 
ily synthesized from research in linguistics. The 
classification work is strictly grounded in syntac¬ 
tic properties of the verbs and was not targeted a 
priori at modal meanings. 

4 Evaluation 

4.1 Linking to Semantic Classes 

Our first set of experiments aims at refining fhe 
inifial semanfic characferizafion of fhe classes by 
linking fhem fo independenfly consfrucfed seman¬ 
tic classificalions at the word sense level. Specif¬ 
ically, we consider fhree differenl semanfic classi- 
ficafions from compufafional lexicons, which have 
been creafed by linguistic experfs: (i) fhe so-called 
semanfic fields in GermaNef, grouping verb senses 
info 15 coarse classes, such as perception, emo¬ 
tion, (ii) fhe verb classes given in VerbNef, and 
(iii) fhe Frame-semantic frames in FramcNef. As 
fhe GermaNef and FrameNef classes are based 
on differenl lexicographic and linguistic fheories, 
we expecf an addilional semanfic characferizafion 
from fhe linking. The VerbNef classes, which 
also follow Levin’s hypofheses, however, are used 
fo invesfigafe if fhe synfax-semanlics correspon¬ 
dence is mainfained across fhe relafed languages 
German and English. 

For Ibis linking experimenf, we used fhe UBY 
framework (Gurevych el ah, 2012)^°, conlaining 
slandardized versions of fhe above lexicons, as 
well as a linking belween VerbNef and FrameNef 
on fhe word sense level. 

Approach In order fo link our classes fo verb 
senses in GermaNef and VerbNef, we developed 
an aulomalic linking melhod based on subcal 
frame similarity. Recognizing subcal frame sim¬ 
ilarity requires a common slandardized formal for 
fhe olherwise incomparable frames. UBY pro¬ 
vides such a slandardized formal which has been 
presented in delail by Eckle-Kohler and Gurevych 
(2012). If represenls subcal frames uniformly 
across German and English, and af a fine-grained 
level of individual synlaclic argumenls. Our link¬ 
ing approach is based on fhe following hypolhe- 
sis: Two verb senses wilh equivalenl lemmas are 
equivalenl, if Ihey have similar subcal frames.^' 
Our melhod inlerprels fhe pairs of verb and sub- 

'°http://WWW.ukp.tu-darmstadt.de/uby/ 

"This approach is applicable for GermaNet, because Ger- 
maNet contains fine-grained syntactic subcat frames. 


cal frame listed in our classification^^ as senses. 
While we do nof claim lhal Ihis hypolhesis is suf- 
ficienf in general, i.e., for all verb senses, we found 
lhal if is valid for fhe subsel of senses belonging fo 
fhe class of AO-selecling verbs. 

The cross-lingual linking of our classes fo Verb- 
Nel senses requires an addilional Iranslafion sfep, 
which we describe firsl. 

Manual Translation While UBY also provides 
Iranslalions belween German and English verb 
senses, e.g., as pari of fhe Interlingual Index from 
EuroWordnel (lEI), we found lhal many of fhe 
Iranslalions were nof presenl in our largel lexicon 
VerbNef. Therefore, fhe main aulhor of Ihis pa¬ 
per, a nalive speaker of German wilh a good pro¬ 
ficiency in English, Iranslaled fhe AO-compalible 
verbs (i.e., word senses) manually using Einguee'^ 
and diclionaries. This look aboul 7 hours. 

Eor 23 German verbs, we could nof find any 
equivalenl lexicalized Iranslafion, because Ihese 
verbs express very fine-grained semanfic nuances. 
Eor example, we did nof find an equivalenl English 
verb for a few verbs in fhe aspeclual class buf only 
a Iranslafion consisting of an adjeclive in combina- 
lion wilh to be. Examples include be easy (leicht- 
falien), be willing (sich bereitfinden), be capable 
(vermdgen), which have German equivalenls lhal 
are lexicalized as verbs. As a resull, we arrived 
al Iranslalions for 614 oul of 637 German verbs. 
These 614 German verbs are Iranslaled fo 413 En¬ 
glish verbs, indicating lhal fhe English Iranslafion 
has a more general meaning in many cases. 

Automatic Verb Sense Linking Our algorilhm 
links a German verb sense (or ils English fransla- 
lion) wilh a GermaNef (or VerbNef) sense, if fhe 
subcal frames of bolh verb senses have fhe same 
number of argumenls and if fhe argumenls have 
cerlain fealures in common. Eor example, fo cre- 
ale a link fo GermaNef, fealures such as fhe com- 
plemenlizer of clausal argumenls and fhe case of 
noun phrase argumenls have lo agree. In a similar 
way, Ihe linking lo VerbNel is based on a compar¬ 
ison of German subcal frames and English subcal 

*^We consider only verb senses that are compatible with 
AOs, as indicated by subcat frames with clausal or non-finite 
arguments. 

'^Linguee (http : / /www. linguee . de/) is a transla¬ 
tion tool combining an editorial dictionary and a search en¬ 
gine processing bilingual texts. In particular, it provides a 
large variety of contextual translation examples. 

'■^We do not link the subcat frames, but we do compare 
them across the related languages German and English to de¬ 
termine their similarity in the context of linking. 




frames - which are represented uniformly across 
German and English. In Section A.2, we provide 
more details about the algorithm. 

Results According to a manual evaluation of a 
random sample of 200 sense pairs, the automatic 
verb sense linking yielded an accuracy of 89.95% 
for the linking to GermaNet, and 87.54% for the 
linking to VerbNet {k agreement on the sample 
annotated by two annotators was 0.7 and 0.8, re¬ 
spectively). The main types of errors in the linking 
to GermaNet and VerbNet are due to specific syn¬ 
tactic features of the subcat frames which diverge 
and are not considered in the automatic linking. 
The differences regarding these specific feafures 
are due fo cross-lingual differences (VerbNef, e.g., 
verb phrase argumenfs wifh ing-form) and diverg¬ 
ing linguistic analyses of parficular consfrucfions 
(GermaNef, e.g., consfrucfions wifh es (it)), see 
also Eckle-Kohler and Gurevych (2012). 

By linking fhe verbs in our classification 
fo semantic classes in GermaNef, VerbNef and 
ErameNef, we obfain a fhree-way semantic char- 
acferizafion of our classes. The linking fo fhe 
GermaNef semantic fields covers 270 (43%) of 
fhe source verbs. Of fhese, 219 (81%) are linked 
fo fhe fhree semanfic fields cognition, communi¬ 
cation and social. Eewer verbs (32 (12%)) are 
linked fo fhe semantic fields emotion, percep¬ 
tion, change. Semantic fields nol among fhe far- 
gef classes are consumption, competition, contact, 
body and weather. 

Table 2 summarizes fhe linking fo VerbNef and 
ErameNef and shows how many verbs from each 
source class could be linked fo any of fhe classes in 
VerbNef or ErameNef.As fhe class disfribufion 
of fhe verb subsefs covered by our linking-based 
evaluafion is similar as for fhe original classes, we 
consider our evaluation as valid, allhough less fhan 
50% of all verbs could be evaluated Ibis way. 

The largel classes in VerbNef and ErameNef re¬ 
veal meaning componenfs fhaf are on fhe one hand 
unique for individual classes, and on fhe ofher 
hand shared across several German classes. 

The future-oriented class confains objecf con- 
frol verbs (e.g., force-59, forbid-67 in VerbNef, 
and request, preventing in ErameNef). The wh/if- 
factual class is unique regarding fhe cognition and 
perception verbs (e.g., discover-84, see-30.1-1, 
and perception-experience). The future-directed 

'^Based on the percentage of source class members linked 
to any of the target classes, we only display target classes with 
an overlap of at least 1.8% due to space constraints. 


Verb class | Wiki Web | News || News Eng. 


all 

25.85 

50.58 

33.91 

25.31 

aspectual 

0.90 

0.80 

1.44 

1.96 

future-oriented 

9.45 

23.04 

13.65 

12.58 

interrogative 

0.01 

0.05 

0.05 

0.65 

wh-factual 

4.26 

17.89 

4.99 

3.48 

fo. wh-factual 

0.29 

0.28 

0.85 

1.14 

wh/if -factual 

3.02 

2.54 

3.53 

5.20 

fo. wh/if-factual 

2.36 

1.77 

3.14 

5.75 

non-factual 

4.29 

3.36 

4.84 

3.57 


Table 3: Percenfage of classes in corpora: German 
Wikipedia (Wiki), SDeWaC (Web), Tiger (News); 
English Reufers corpus (News Eng.). 

wh/if-factual class also confains objective assess¬ 
ment verbs, as shown by fhe esfimafe-34.2 class. 
The verbs in fhe fwo wh-factual classes share 
meaning componenfs as well, as shown by fhe 
opinion verb classes admire-31.2 and defend-85 in 
VerbNef ov judgment, tolerating in ErameNef. 

While fhere are also ofher VerbNef and 
ErameNef classes shared across several classes, 
fhey fumed ouf fo be very general and underspec¬ 
ified regarding fheir meaning, fhus nol conlribul- 
ing fo a more fine-grained semanfic characleriza- 
lion. Eor example, fhe conjeclure-29.5 class as¬ 
sembles quite diverse conjecfure verbs, e.g. verbs 
expressing opinion (feel, trust) and faclualily (ob¬ 
serve, discover). A similar observation holds for 
fhe statement frame in ErameNef. 

4.2 Analysis of Frequency and Polysemy 

In order fo assess fhe usefulness of fhe verb re¬ 
source for NEP fasks, we defermined fhe lemma 
frequency of all verbs in fhe 8 classes in SDeWaC 
(EaaB and Eckarf, 2013), a cleaned version of fhe 
German DeWaC corpus (Baroni and Kilgarriff, 
2006). A ranking of fhe verbs according fo fheir 
lemma frequency showed fhaf 89% of fhe verbs 
occur more fhan 50 times in SDeWaC. 

We also analyzed fhe frequency disfribufion of 
fhe 8 verb classes in fwo ofher German corpora 
belonging fo differenl genres, and also for En¬ 
glish, see Table 3:^^ encyclopedic fexf (fhe Ger¬ 
man Wikipedia^*), German newspaper fexf (fhe 
Tiger corpus (Branis el al., 2004)), and fhe English 

*®In the verb resource we provide for download, we 
included this frequency information in order to enable 
frequency-based filtering. 

'^Details of the computation of the verb lemma frequency 
lists are given in the appendix A.l. 

'*www. Wikipedia . de, dump of 2009-06-18 



Reuters-21578 eorpus.^^ Table 3 shows that the 
large verb elasses eonstitute a substantial propor¬ 
tion of verb oeeurrenees aeross different genres. 
This suggests that the verb elasses might be useful 
features for various text elassifieation tasks. 

We performed a further analysis of the poly¬ 
semy of the German and English verbs in our 
elasses relative to several fine and eoarse word 
sense inventories. Regarding GermaNet, there 
are 2.28 senses per verb (1.53 for all GermaNet 
verbs), whereas WordNet lists 5.11 senses per 
verb (2.17 for all WordNet verbs). In VerbNet, 
we find 1.74 senses per verb (1.42 for all Verb- 
Nef verbs), and in FrameNef 1.96 (1.52 for all 
FrameNef verbs). This analysis shows fhaf fhe fask 
of aufomafie sense linking is parfieularly hard for 
fhe eafegory of AO-seleefing verbs we eonsider. 
Whefher fhe polysemy is an issue for any appliea- 
fion where fhe verb elasses are used as feafures is 
nol a priori elear and depends on fhe fask af hand. 

4.3 Textual Entailment Experiment 

For an exfrinsie evaluation, we investigated the 
usefulness of the German and the English verb 
elasses as features in reeognizing textual entail¬ 
ment (RTE). In RTE, the task is to determine 
whether for a pair of text fragments - the text T 
and the hypothesis H - the meaning of H is en¬ 
tailed by T (Dagan et ah, 2006); for non-entailing 
pairs, sometimes a further eategory “unknown” is 
used as a label. 

We employed a simple elassifieation-based ap- 
proaeh to RTE and trained and evaluated a Naive 
Bayes elassitier on the test sets of three RTE 
benehmarks, using 10-fold eross validation: the 
English RTE-3 data (Giampieeolo et ah, 2009) and 
their German translation^^ (the development sets 
and the test sets eaeh eonsist of 800 pairs), and an 
expanded version of the English RTE-3 data from 
the Sagan Textual Entailment Test Suite (Castillo, 
2010) eonsisting of 2974 pairs. While the Ger¬ 
man dataset provides a two-way elassifieation of 
the T-H pairs, the two English datasets provide a 
three-way elassifieation, also using the “unknown” 
label. We used the DKPro TC framework (Daxen- 
berger et al., 2014) for elassifieation and applied 
POS tagging and lemmatization as preproeessing. 

'^Reuters-21578, Distribution 1.0, see http:// 
kdd.ics.uci.edu/databases/reuters21578/ 
reuters21578.html. 

^"http://www.dfki.de/-neumann/ 
resources/RTE3_DE_V1.2_2013-12-02.zip 


As a baseline feature, we use the word over¬ 
lap measure between two T-H pairs (no stopword 
filtering, no lemmatization, no normalization of 
overlap seore), whieh is quite eompetitive on the 
RTE-3 data, beeause this dataset shows a high 
differenee in word overlap between positive (en¬ 
tailment) and negative (no entailment) pairs (Ben- 
tivogli et al., 2009). 

An analysis of the development set of the Ger¬ 
man RTE-3 data showed that 62% of the pairs eon- 
tain at least one oeeurrenee of any of the verbs 
from the elassifieation in either T or H. However, T 
and H fragments display no statistieahy signiheant 
differenees^^ regarding the oeeurrenees of any of 
the verb elasses. 

A detailed analysis revealed that pairs without 
entailment are often eharaeterized by a mismatch 
between T and H regarding the presence of factu- 
ality markers. For example, the presence of verbs 
indicating uncertainty (all classes apart from wh- 
factual and wh/if-factual) in T and an absence of 
such verbs in H might indicate non-entailment as 
in the following not entailing pair from the English 
RTE3 development set where “long” signals non- 
factuality, but “researching” signals factuality: 

T: The BBC’s Americas editor Will Grant says 
many Mexicans are tired of conflict and long 
for a return to normality. 

H: Will Grant is researching a conflict with 
Mexicans. 

Thus, an insufficient overlap of modality markers 
in T and H might actually indicate non-entailment, 
but lead to an incorrect classification as entailment 
when considering only word overlap. 

Accordingly, we implemented a factuality- 
mismatch feature both for German and for En¬ 
glish, based on our new German and English 
classes. This feature is similar to the word 
overlap feature but with lemmatization and nor¬ 
malization of overlap score. Verb class counts 
are based on verb lemma counts of the mem¬ 
ber verbs; for English verbs that are members 
of more than one class, we included all verb 
classes in our factuality-mismatch feature.^^ Ta¬ 
ble 4 shows the results. While the differences 

All significance scores in this paper are based on Fisher’s 
exact test at significance level p<0.05. 

^^In the German part, every verb is assigned to one class, 
while the translation to English resulted in 22% of the English 
verbs being members in more than one class. However, only 
11% of the multiple class assignments involve a combination 
of factual and uncertainty classes. 



for RTE-3 DE and RTE-3 EN are not statisti¬ 
cally significant, the factuality-mismatch feature 
yielded a small but significant improvement on 
the expanded RTE-3 EN dataset. This is due to 
the different nature of the expanded RTE dataset, 
which was created using a paraphrasing technique. 
As a result, the number of occurrences of verbs 
from our classes increased, and the factuality- 
mismatch became a discriminative feature for 
distinguishing between CONTRADICTION and 
UNKNOWN/ENTAIEMENT. 

Considering the fact that we employed only 
simple overlap features that do not rely on depen¬ 
dency parsing and did not perform any word sense 
disambiguation, these results suggest that the verb 
classes might be promising features for RTE, both 
for German and English. As factuality can be ex¬ 
pressed by a variety of further linguistic means, 
including modal verbs, negation, tense and certain 
adverbs, investigating the combination of our verb 
classes with other modality signals might be espe¬ 
cially promising as part of future work. 



RTE-3 DE 

RTE-3 EN 

RTE-3 EN exp. 

WO 

59.87 

54.75 

54.98 

WO+FM 

59.25 

54.62 

58.81 


Table 4: Accuracy of a Naive Bayes classifier (10- 
fold cross validafion on fhe fesf sefs) wifh word 
overlap (WO) and addifional facfualify-mismafch 
(WO-I-EM) feafures. 

5 Results and Discussion 

Our consfrucfion of semantic classes from fhe syn- 
facfic behavior of AO-selecfing verbs resulfs in an 
invenfory of modal meanings fhaf emerged from a 
large lexical resource. The main resulf of fhe link¬ 
ing based evaluation is a defailed semantic char- 
acferizafion of fhe inferred classes - a prerequisite 
for using fhem in NEP fasks in an informed way. 
The semantic classes seem fo be parficular suifed 
for fasks relafed fo opinion analysis, fexfual infer¬ 
ence, or argumenfafion mining. In fhis confexf, fhe 
relafionship befween our large resource of lexical 
verbs and fhe closed class of modal verbs mighf be 
an inferesfing question for fufure research. 

Mosf of all, fhe linking fo GermaNef and 
ErameNef shows fhaf if is indeed possible fo nar¬ 
row down meaning componenfs for Eevin classes. 
Moreover, fhe resulfs of fhe linking fo Verb- 
Nef also provide supporf for Eevin’s hypofhesis 


fhaf fhe correspondences befween verb synfax and 
meaning described for English largely apply fo fhe 
relafed language German as well (Eevin, 2015b). 

The English version of fhe semantic classes 
which we created by means of franslafion has fhe 
same semantic properfies as fhe German classes. 
However, fhe synfacfic properfies of fhe English 
classes are nof fully specified, because English has 
addifional kinds of non-finife argumenfs, such as 
ing-forms or bare infinitives. Therefore, if mighf 
be inferesfing fo address fhis question in fhe fu¬ 
fure and fo build a similar semantic classification 
for English from scrafch, in parficular in fhe con¬ 
fexf of exfracfing modalify classes from corpora. 
This would require an adapfafion of fhe synfacfic 
signafures, considering fhe various kinds of non- 
finife argumenfs parficular fo English. Based on 
large subcafegorizafion lexicons available for En¬ 
glish (e.g. COMEEX (Grishman ef ah, 1994) or 
VerbNef), if should be feasible fo derive such sig¬ 
nafures and fo consfrucf a mapping of signafures fo 
modalify aspecfs in a similar way as for German. 

The quesfion whefher fhe synfacfic signafures 
can be recovered in large corpora is parficularly in¬ 
feresfing, because fhis would allow exfending fhe 
exisfing classes and fo also acquire AO-selecfing 
adjecfives and nouns. We plan fo invesfigafe fhis 
quesfion as parf of fufure work. 

6 Conclusion 

We inferred semanfic classes from a large syn¬ 
facfic classificafion of German AO-selecfing verbs 
based on findings from formal semanfics abouf 
correspondences befween verb synfax and mean¬ 
ing. Our fborough evaluafion and analysis yields 
defailed insighfs info fhe semanfic characferisfics 
of fhe inferred classes, and we hope fhaf fhis al¬ 
lows an informed use of fhe resulfing resource in 
various semanfic NEP fasks. 
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Sense linking based on subcategorization frames 

get lexical entry ICa of source verb Va 
get equivalent verb vt in target lexicon 
get lexical entry let of target verb vt 
forall frame f in ICa 
get listOfArguments h of ft 
forall frame fj in let 
get sense sj of frame fj 
get listOfArguments Ij of fj 
if size{li) = size{lj) 

AND features{li) = features{lj) 
link [va, fi) and Sj 
end if 
end for 
end for 


Table 5: Algorithm for verb sense linking. 


A.2 Verb Sense Linking 

For the linking-based evaluation, we used UBY 
(version 0.7.0) versions of the following three re- 
sourees: the German wordnet GermaNet (version 
9.0), the English lexieons VerbNet (version 3.2) 
and FrameNet (version 1.5). 

The algorithm for eross-lingual verb 
sense linking is given in pseudo-eode 
in Table 5. The implementation is 

available at github.com/UKPLab/ 
acl2016-modality-verbclasses. 


A Supplemental Material 

A.l Verb Lemma Frequency List 

In order to eount the oeeurrenees of verb lem¬ 
mas in the German eorpus SDeWaC, we used 
a reader and pre-proeessing eomponents (i.e., 
the LanguageTool segmenter and the TreeTag- 
ger for POS tagging and lemmatization) from 
the DKPro Core eoheetion (Eekart de Castilho 
and Gurevyeh, 2014). From DKPro Core, 
we also used a eomponent that deteets sepa¬ 
rated partieles of German partiele verbs and re- 
plaees the lemma of the verb base form anno¬ 
tated by the TreeTagger by the true lemma of 
the partiele verb. Our verb lemma eounting 
pipeline is available at github. com/UKPLab/ 
acl2 016-modality-verbclasses. 



